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Abstract. A statistical metliod for reconstructing large scale structure 
behind the Zone of Avoidance is presented. It also corrects for shot-noise 
and for redshift distortion in galaxy surveys. The galaxy distribution is 
expanded in an orthogonal set of spherical harmonics. We show that in 
the framework of Bayesian statistics and Gaussian random fields the 4ir 
harmonics can be recovered and the shot-noise can be suppressed, giving 
the optimal picture of the underlying density field. The correction factor 
from observed to reconstructed harmonics turns out to be the well-known 
Wiener filter (the ratio of signal to signal-|-noise) , which is also derived by 
requiring minimum variance. We apply the method to the 1.2 Jy IRAS 
survey. A reconstruction of the projected galaxy distribution confirms 
the connectivity of the Supergalactic Plane across the Galactic Plane (at 
Galactic longitude / ~ 135° and / ~ 315°) and the Puppis cluster behind 
the Galactic Plane (/ ~ 240°). The method is extended to 3-D, and is 
used to recover from the 1.2 Jy redshift survey the density, velocity and 
potential fields in the local universe. 



1. Introduction 

Where the Zone of Avoidance (ZOA) cannot be observed directly, the alternative 
is to reconstruct the structure in a statistical way. Previous corrections for 
unobserved regions in catalogues were done, somewhat ad-hoc, by populating the 
ZOA uniformly according to the mean density, or by interpolating the structure 
below and above the Galactic Plane (e.g. Lynden-Bell et al. 1989, Yahil et al. 
1991, Strauss et al. 1992, Hudson 1993). 

Two other problems appear in analysing the distribution of galaxies. First, 
if one assumes that the distribution of luminous galaxies samples an underlying 
smooth density field, then the discreteness of objects introduces Poisson 'shot- 
noise'. Second, due to peculiar velocities, redshift surveys give a distorted picture 
of the density field. Here we show how to recover all-sky projected density field, 
from galaxy surveys which suffer incomplete sky, and also how to reconstruct the 
density, velocity and potential fields from redshift surveys. The results presented 
here are based on several recent studies, by Lahav et al. (1994), Fisher et al. 
(1994b) and Zaroubi et al. (1994). Hoffman (this volume) discusses other aspects 
of the method, mainly when applied in Cartesian coordinates, and Bunn et al. 
(1994) recently applied a similar method to reconstruct the COBE DMR map. 
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The recovery of a signal from noisy and incomplete data is a classic problem 
of inversion, common in problems of image processing (e.g. 'seeing' or blurred 
HST images). A straightforward inversion is often unstable, and a regularization 
scheme of some sort is essential in order to interpolate where data are missing or 
noisy. By avoiding any prior assumptions one allows the noise and incomplete 
data to sometime dominate the resulting reconstruction. In the Bayesian spirit 
we use here raw data and a prior model to produce 'improved data'. This may 
raise the question to what extent a reconstruction of say the Great Attractor 
depends on what is assumed about the unknown nature of the spectrum of 
fluctuations. But the prior model does not necessarily require a speculative 
assumption. In the context of this work we simply require a reconstruction 
which obeys the constraint of the 2-point correlation function of the observed 
galaxy distribution, as derived from a smaller section of the sky. Using the above 
principles we derive a Wiener filter (the ratio of signal to signal+noise) , which 
also follows from requiring minimum variance (e.g. Rybicki & Press 1992). 



2. Wiener filter in theory and in practice 

Let us first consider a simple pedagogical example. Assume two Gaussian vari- 
ables, X and y, with zero mean, (a;) = (y) = (hereafter (...) denote ensemble 
average) . The probability for x given y is by the rule of conditional probability 
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where for Gaussian probability P{y) oc exp (— 2<^2^ ); and the joint probability 
is a bivariate Gaussian P[x^y) oc exp [—\{u^ — 2puv + f^)/(l — /O^)], where 
u = x/{x'^y^'^, V = y/{y'^y^'^ and p = {xy) / \/{x'^){y'^) . It then follows that the 
conditional probability is simply a 'shifted Gaussian' 

P{x\y) a exp [-^{u - pvf/{l - p% (2) 



The maximum a poteriori probability clearly occurs for u = pv, ov x = -jp^ y. 
In the special case of Gaussian fields the most probable reconstruction is also 
the mean field (cf. Hoffman & Ribak 1991; Kaiser & Stebbins 1991). Hereafter 
we term them together as the 'optimal reconstruction'. 

Exactly the same result for the 'optimal reconstruction' is also obtained 
by a different approach, by asking for the linear filter F which minimizes the 

variance {{x — FyY). Minimizing with respect to F gives indeed F = -jp^ 

and X = Fy, as above. Note that although the results of the two approaches 
are identical, due to the quadratic nature of the functions and the linearity 
of the filter, the underlying assumptions are quite different. The conditional 
probability approach (eq. 1) requires to specify the full distribution functions 
(Gaussians in our case). On the other hand, the minimum variance approach 
only considers the second moment of the distribution function, but assumes a 
linear filter F. 
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Consider now the special case tliat y = x + a , wliere cr is a Gaussian noise 
uncorrelated witli tlie true signal x (hence (scr) = 0). It follows that the optimal 
estimator of the signal x given the (noisy) measurement y is 



The factor {F) in front of the measurement y is the well-known Wiener filter 
commonly used in signal processing (Wiener 1949; for review see e.g. Press et 
al. 1992, Rybicki & Press 1992). Note that it requires a priori knowledge of the 
variances in the signal and the noise. When the noise is negligible the factor 
approaches unity, but when it is significant the measurement is attenuated. 

A third approach is of adding a regularizing function to the usual (log- 
Likelihood) minimization. In fact a regularization function of the form x"^ , moti- 
vated in our case by physical considerations of the underlying field, yields essen- 
tially a Wiener filter. Other reconstruction method use different regularization 
functions, e.g Maximum Entropy (e.g. Gull 1989) takes x\nx. 

Here we have only considered a simple example of two variables. More 
generally, for vectors of signal and noise and a response function Wap (e.g. a 
'point spread function') one can write y„ = Wap[xp + ap\ and derive a Wiener 

solution of the form x^ = (xay-y) (2/7?/^)"^ Up- The Wiener formulation is greatly 
simplified by using orthogonal set of functions, e.g. by employing a Fourier or 
harmonic transforms. In particular, if Wap = 1 then the Wiener matrix in 
Fourier space is diagonal, and eq. (3) holds, but with the variables replaced by 
their Fourier transforms. 

Figure 1 shows a 1-d example, which is also of relevance to the ZOA problem. 
The solid line at the bottom panel is a mock 'double-horned' HI spectrum of 
a galaxy (generated by H. Ferguson). To this we added real noise taken from 
the Dwingeloo radio-telescope, resulting in a noisy spectrum at the top panel. I 
then applied a Wiener filter in Fourier space, using the prior rms of the galaxy 
and the noise (here of course we know what they are). The dotted line in the 
bottom panel shows the Wiener reconstruction which indeed recovers reasonably 
well the galaxy spectrum. In fact, we are developing this Wiener approach as 
a detection algorithm for the Dwingeloo project of HI blind search behind the 



3. Expansion in Spherical Harmonics 

Back to the large scale structure, in analysing galaxy surveys the most infor- 
mative data set is of course the catalogue itself. However, it is more efficient 
and sometimes more insightful to compress the galaxy data. Here we use spher- 
ical harmonics to expand the galaxy distribution in a whole-sky survey. This 
technique has been considered for 2-D samples (e.g. Peebles 1973, Scharf et al. 
1992) and more recently for analysing redshift and peculiar velocity surveys (e.g. 
Regos & Szalay 1989; Scharf & Lahav 1993; Lahav et al. 1993; Fisher, Scharf & 
Lahav 1994a; Nusser & Davis 1994; Lahav 1994 for a summary of properties). 
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Figure 1. HI spectrum of a mock galaxy+noise (top panel) and its 
Wiener reconstruction (dotted line, bottom panel), compared with the 
original mock galaxy spectrum (solid line, bottom line). The units are 
arbitrary. 
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In projection, tlie density field over 4ir is expanded as a sum: 



m = -\-l 

J2 J2 



(4) 



wliere tlie Y/m's are tlie ortlionormal set of splierical liarmonics : Yifn{0,(f)) oc 

P;'™' (cos^) exp(im(^), wliere 6 and (f) are tlie spherical polar angles, and P^'™''s 
are the associated Legendre Polynomials of degree / and order m. The spherical 
harmonic analysis provides a unified language to describe the local cosmography 
as well as the statistical properties (e.g. the power-spectrum) of the galaxy 
distribution. In particular it retains both the amplitude and phase information 
and hence the underlying texture of the distribution. 

The mean-square of harmonics can be related to the power-spectrum of 
mass fiuctuations Pm{k) = in Fourier space. In particular, the variance in 

harmonics measured in a fiux limited redshift survey can be formulated (Fisher, 
Scharf & Lahav 1994a) assuming linear theory as 



l«LP> 



vl/f(fc) + ^vl;-(fc) 



0.6 



(5) 



are 'window functions'. By applying this relation to 



the 1.2 Jy redshift survey (of Fisher 1992, Strauss et al. 1992) we find for the 
combination of density and bias parameters, Q^'^ /h ~ 1.0 ± 0.3 (assuming the 
observed IRAS galaxy power-spectrum) 



4. Mask inversion using Wiener filter 

We turn now to the problem of reconstructing large scale structure behind the 
ZOA. The problem is formulated as follows: What are the full-sky noise-free 
harmonics given the observed harmonics, the mask describing the unobserved 
region, and a prior model for the power-spectrum of fiuctuations ? 

The observed harmonics cim,obs (with the masked regions filled in uniformly 
according to the mean) are related to the underlying 'true' whole-sky harmonics 
(ilm by (cf. Peebles 1980, eq. 46.33) 

cim,obs = J2J2 [«''™' + (6) 

/' m' 

where the monopole term (/' = 0) is excluded. We have added the shot-noise 
(Ta in the 'true' number- weighted harmonics a/m's (not in the c/^'s). The noise 
variance is estimated as (cr^) = M (the mean number of galaxies per stera- 
dian, independent / in this case). The harmonic transform of the mask, W]]]"^ , 
introduces 'cross-talk' between the different harmonics. 

By analysis similar to that given in section 2 it can be shown (Lahav et al. 
1994; Zaroubi et al. 1994) that the solution of this inversion problem is 



a = FW-i Cobs, 
5 



(7) 



where the vectors a and Cgbs represent the sets of observed harmonics {a/m} and 
{c/m,o6s}j with the diagonal Wiener matrix 

Here {af)th is the cosmic variance in the harmonics, which depends on the power- 
spectrum (cf. eq. 5). We emphasize again that in the special case of underlying 
Gaussian field the most probable field, the mean field and the minimum variance 
Wiener filter are all identical. The scatter in the reconstruction is at least as 
important, and it can also be written analytically for Gaussian random fields. 

Even if the sky coverage is 4ir (W = I), the Wiener filter is essential to 
reveal the optimal underlying 'continuous' density field, cleaned of noise. In 
the absence of other prior information on the location of clusters and voids, the 
correction factor is 'isotropic' per /, i.e. independent of m, so in the case of full 
sky coverage, only the amplitudes are affected by the correction, but not the 
relative phases. For example, the dipole direction is not affected by the shot- 
noise, only its amplitude. But of course, if the sky coverage is incomplete, both 
the amplitudes and the phases are corrected. The reconstruction also depends 
on number of observed and desired harmonics. Note also that the method is 
non-iterative. Since the Wiener factor is less than unity, applying it iteratively 
will result in zero signal ! 

5. Reconstruction of the projected IRAS 1.2 Jy galaxy distribution 

Here we apply the method to the sample of IRAS galaxies brighter than 1.2 Jy 
which includes 5313 galaxies, and covers 88 % of the sky. This incomplete sky 
coverage is mainly due to the Zone of Avoidance, which we model as a 'sharp 
mask' at Galactic latitude |5| < 5° . The mean number of galaxies is Af = 392 per 
steradian, which sets the shot-noise, (cr^). As our model for the cosmic scatter 
{af)th we adopt a fit to the observed power spectrum of IRAS galaxies (Fisher 
et al. 1993). 

Figure 2 shows the reconstruction of the projected IRAS 1.2 Jy sample. 
The Zone of Avoidance was left empty, and clearly it 'breaks' the possible chain 
of the Supergalactic Plane and other structures. Figure 3 shows our optimal 
reconstruction for 1 < / < 15. Now the structure is seen to be connected 
across the Zone of Avoidance, in particular in the regions of Centaurus/Great 
Attractor (/ ~ 315°), Hydra (/ ~ 275°) and Perseus-Pisces (/ ~ 315°), confirming 
the connectivity of the Supergalactic Plane. We also see the Puppis cluster 
(/ ~ 240°) recovered behind the Galactic Plane. This cluster has been noticed 
in earlier harmonic expansion (Scharf et al. 1992) and other studies (Lahav et 
al. 1993 and references therein; Yamada in this volume). The other important 
feature of our reconstruction is the removal of shot noise all over the sky. This 
is particularly important for judging the reality of clusters and voids. 

Comparison of our reconstruction with the one applied (using a 4ir Wiener 
filter) to the IRAS sample in which the ZOA was filled in 'by hand' across the 
Galactic Plane (Yahil et al. 1991) shows good agreement. We have also used 
other prior realistic models and found that the reconstructions changed very 
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Figure 2. Harmonic expansion (1 < / < 15) of tlie projected raw 
IRAS 1.2 Jy data in Galactic AitofF projection. Regions not observed, 
in particular |5| < 5 (marked by dashed lines), were left empty. The 
contour levels of the projected surface number density are in steps of 
100 galaxies per steradian (the mean projected density is A/" ~ 400 
galaxies per steradian). 
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Figure 3. A whole-sky Wiener reconstruction of figure 2. The re- 
construction corrects for incomplete sky coverage, as well as for the 
shot-noise. The reconstruction indicates that the Supergalactic Plane 
is connected across the Galactic Plane at Galactic longitude / ~ 135° 
and / ~ 315°. The Puppis cluster stands out at the Galactic Plane at 
/ r~j 240°. The horizontal dashed lines at 5 = ±5° mark the major Zone 
of Avoidance in the IRAS sample. The contour levels are as in Figure 
2. (From Lahav et al. 1994.) o 



little. As a more challenging test of the method we have also used an iV-body 
simulation of standard Cold Dark matter (where the whole 'sky' true harmonics 
are known) and varied the size of the ZOA. We find that for mask larger than 
|5| = 15° it is difficult to recover the unobserved structure. In this case extra- 
regularization is required, e.g. by truncating components in the Singular Value 
Decomposition (Press et al. 1992; Lahav et al. 1994). Clearly the success of 
the method depends on the interplay of three angular scales: the width of the 
mask, the desired resolution (ir/lmax) and the physical correlation of structure. 

6. 3-D reconstruction of density, velocity and potential fields 

To extend the reconstruction method for analyzing redshift surveys we expand 
the fiuctuations in the density field in spherical harmonics Y/m and spherical 
Bessel functions ji{z) (cf. Binney & Quinn 1991, Lahav 1994, Fisher et al. 
1994b) : 

p{r) = ^^/>/m(r)Y/m(r) = J2 J2 P^"^"^ jli^nr) Y/m(r) , (9) 

I m I m n 

The harmonics and Bessel functions are natural for this problem as they are 
the eigen-functions of Poisson equation, and provide a convenient framework for 
dynamical calculations. The C„;'s define the normalization. 

We shall assume that the data are given within a sphere of radius R, such 
that inside the sphere the desired density fiuctuation is specified by pim{r), but 
for r > R the fiuctuation is pim{i') = (this simply refiects our ignorance about 
the density field out there; the fiuctuations do not vanish of course at large 
distances). The Fourier A;„'s are chosen to ensure orthogonality of the Bessel 
functions, by imposing as boundary condition that the logarithmic derivative of 
the potential is continuous at r = i?. 

An estimator for the density field from the redshift survey is 

pLu = T7~r jiC^ns) i"4(r), (10) 

gal ' 

where the sum is over galaxies with r < R, and (f){r) is the radial selection 
function. 

Assuming 4ir coverage, two corrections are needed in order to convert the 
redshift space coefficients to noise-free coefficients in real space pf^^- It is shown 
in detail in Fisher et al. (1994b) that this can be done by first correcting the 
density coefficients in redshift space for the distortion (assuming linear the- 
ory and OP'^/b; cf. eq. 5), and then applying a Wiener filter to remove the 
shot-noise, assuming a prior for the power-spectrum. Armed with the density 
coefficients one can then predict (using linear theory) the peculiar velocity field 
due to the mass distribution represented by galaxies inside the spherical volume. 
The method provides a non-parametric description of the density, velocity and 
potential fields, which are related by simple linear transformations. Figure 4 
shows a reconstruction by this method using the IRAS 1.2 Jy redshift survey, a 
prior IRAS power-spectrum and OP'^ /b = 1. Here we have used the IRAS survey 
with the ZOA filled in by the interpolation of Yahil et al. (1991), which gave 
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Figure 4. (a) The reconstructed real space density field from the 
1.2 Jy IRAS redshift survey in the Supergalactic Plane. Contours are 
spaced at = 0.5 with solid (dashed) lines denoting positive (neg- 
ative) contours. The heavy solid contour corresponds to ^ = 0. (b) 
Reconstructed dimensionless gravitational potential field, (^(r)/c^ from 
the 1.2 Jy survey for /3 = 1. Contours are spaced at A(^/c^ = 5 X 10~^. 
Solid (dashed) contours denote positive (negative) values with the 
heavy contour representing = 0. (c) Reconstructed radial velocity 
field. Closed (open) dots represent positive (negative) velocities, (d) 
Reconstructed three dimensional peculiar velocity field. (From Fisher 
et al. 1994b.) 



10 



similar results to our Wiener projected reconstruction described above. While 
the 3-D method can be extended to account for the incomplete sky coverage, it 
was more convenient mathematically in this case (with relatively small of ZOA 
and harmonics / < 15) to use the interpolated data and to formulate the prob- 
lem for 4ir. It is remarkable that the two major dynamical features in the map, 
the Great Attractor region (at SGX ~ —3500 km/sec; SGY ~ km/sec in 
the density map) and the Perseus-Pisces supercluster (at SGX ~ 5000 km/sec; 
SGY r~j —2000 km/sec) are both very near the ZOA. 

7. Discussion 

We have presented Wiener filtering method for reconstructing the full sky den- 
sity, velocity and potential fields, free of shot-noise. We have also shown that 
a variety of statistical approaches to the problem all lead to the same optimal 
Wiener estimator. The prior assumptions only depend on the observed 2-point 
galaxy correlation function and the nature of the shot-noise. Our assumption 
that the density field is Gaussian and is sampled by luminous galaxies is of 
course only an approximation to the real universe, but it provides a convenient 
framework which can be further extended. This method is to be applied to new 
all-sky IRAS and optical redshift surveys, and to surveys of the peculiar veloc- 
ity field. The pimn^ coefficients allow objective (non-parametric) comparison of 
different surveys of light and mass in the local universe. As illustrated here this 
reconstruction technique can help in probing the ZOA, and in answering some 
other cosmographic questions. 

Acknowledgments. I thank K. Fisher, Y. Hoffman, D. Lynden-Bell, C. 
Scharf and S. Zaroubi for their contribution to the work presented here and for 
many stimulating discussions. 
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Discussion 

G. Mamon: Can you apply spherical harmonics to one hemisphere of data ? 

O. Lahav: This is not that practical. The Wiener reconstruction can only 
recover structure if the missing zone is relatively small and comparable to the 
correlation scale of the projected galaxy distribution. However, the nice aspect 
of regularization methods such as Wiener or Maximum Entropy is that they can 
tell you 'honestly' if the data are bad or missing, and in this case an area like 
the missing hemisphere is kept 'grey', i.e. at the level of the mean density. 

R. Kraan-Korteweg: If the gap due to lack of data in whole-sky samples is too 
large, your reconstruction method fails. What kind of coverage of the ZOA is 
required to get a reliable reconstruction of the ZOA ? What kind of coverage 
is needed when you expand your method to 3-D and does the Wiener recon- 
struction allow the detection of more detailed structure compared to Potent or 
Hoffman's method which have very large smoothing (1200 km/sec) ? 

O. Lahav: Our experiments with simulations, projected in the same way as the 
IRAS survey, indicate that reliable reconstructions are achieved only if |5| < 15°. 
In 3-D the resolution depends on the number of Bessel radial modes, and this 
choice depends on the selection function, the shot-noise and the scale of non- 
linear redshift distortion. Practically, for current surveys 'optimal' smoothing 
is roughly 500 km/sec (Gaussian half- width) at distance of 4000 km/sec, but it 
increases with distance. 

M. Hendry: You find that the minimum variance and maximum probability 
solutions are equivalent in the Gaussian case. Does this result hold for any 
symmetric probability distribution ? Or, perhaps a more interesting question, 
does the result fail for any non-symmetric distribution ? 

O. Lahav: Indeed the Gaussian case is very special in the sense that the mean 
field is the same as the maximum a posteriori solution, and is also the same as 
the minimum variance solution. This equivalence breaks down for most other 
distribution functions, e.g. for the log-normal function. 

W. Saunders: The Wiener filter leads to unbiased estimates of all the statistical 
quantities of cosmological interest - variances, velocity distortions etc. How- 
ever, on a point to point basis the filtered version is always biased towards the 
mean by an amount dependent on the noise - surely this limits its usefulness for 
cosmographic purposes. 

O. Lahav: First, The Wiener solution, when derived by the minimum variance 
approach, only guarantees this condition. It may, for example, bias the mean 
(this can be cured by subtracting the mean before the reconstruction, and adding 
it back at the end) and high moments. It is true that our formalism assumes that 
we know the noise properties in the rms, not locally. However, the formalism 
can be extended to accommodate other prescriptions. 
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